Naive Bayes vs Logistic Regression: Theory, Implementation and Experimental Validation

On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes

We compare discriminative and generative learning as typified by logistic regression and naive Bayes. We show, contrary to a widelyheld belief that discriminative classifiers are almost always to be preferred, that there can often be two distinct regimes of performance as the training set size is increased, one in which each algorithm does better. This stems from the observationwhich is borne o...

Naive Bayes for Regression

Despite its simplicity, the naive Bayes learning scheme performs well on most classiication tasks, and is often signiicantly more accurate than more sophisticated methods. Although the probability estimates that it produces can be inaccurate, it often assigns maximum probability to the correct class. This suggests that its good performance might be restricted to situations where the output is c...

cumulative logistic regression vs ordinary logistic regression

The common practice of collapsing inherently continuous or ordinal variables into two categories causes information loss that may potentially weaken power to detect effects of explanatory variables and result in Type II errors in statistical inference. The purpose of this investigation was to illustrate, using a substantive example, the potential increase in power gained from an ordinal instead...

Naive Bayes for Regression 3

GENERATIVE AND DISCRIMINATIVE CLASSIFIERS: NAIVE BAYES AND LOGISTIC REGRESSION Machine Learning

This is a rough draft chapter intended for inclusion in the upcoming second edition of the textbook Machine Learning, T.M. Mitchell, McGraw Hill. You are welcome to use this for educational purposes, but do not duplicate or repost it on the internet. For online copies of this and other materials related to this book, visit the web site www.cs.cmu.edu/∼tom/mlbook.html. Please send suggestions fo...